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ABSTRACT 



This paper discusses the rationale for designing better 
literacy assessment for elementary school children. The "Literacy Profile" is 
a literacy assessment procedure designed to collect data about groups of 
elementary students and track literacy development more effectively than 
current standardized tests. It assumes that literacy assessment must: (1) be 

based on competence, (2) reflect what is known about literacy, and (3) foster 
new and different discussions about supporting learners by providing new ways 
to examine children's literate behaviors. To complete a Literacy Profile, 
children peruse collections of books, then select one to read (rather than 
reading contrived passages written for testing) . Research on the Literacy 
Profile involved 55 elementary students and college students who were 
tutoring them. The college students administered the reading interview to 
each child. The activity provided insight into their understanding of 
reading. Next, the children selected books to read aloud onto audiotapes. The 
children also retold each story and wrote and drew about the books. The 
college students recorded and analyzed each reader's miscues . Miscue analysis 
and retelling scores were used to report the extent to which readers were 
effective and efficient. Results suggest the Literacy Profile is more 
authentic than standardized testing because children receive help selecting 
books that interest them (unlike in standardized testing which makes no 
allowances for developmental differences or personal preferences) . (Contains 
44 references) . (SM) 
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This paper discusses the rationale for designing better literacy assessment for 
elementary children and describes the beginning steps toward developing and piloting an 
assessment procedure called “The Literacy Profile”. This work-in-progress presents 
interesting problems to consider and proposes some new ways to think about and assess 
children’s literacy development. The Literacy Profile is being designed as a tool for 
collecting data about large groups of children. If the procedure can be shown to be valid 
and reliable, it may well become a substitute for the standardized tests currently in use in 
schools. 




The Rationale for New Assessments of Literacy 

The construct of literacy has changed dramatically over time. Currently, 
progressive educators view literacy as a process of interpreting the world and developing a 
conscious awareness that values, behaviors, and beliefs are socially constructed (Lytle & 
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Schultz, 1991). This view of literacy is far more complex than thinking of literacy as the 
basic acts of reading and writing. This complex view of literacy demands that we consider 
context, purpose, intertextual features, and multiple meanings to be important to the basic 
acts of reading and writing. At this point in time, we are just beginning to understand the 
constructive nature of reading and its close relationship to writing. We know to watch 
how readers use the multiple cueing systems available in the reading process and to watch 
for the reader’s problem-solving strategies such as rereading, skipping words, or 
substituting meaningful alternatives. We watch how writers interpret tasks and organize 
themselves for writing, as well as their control of the mechanics of language. In short, we 
beginning to think more complexly about the processes of reading and writing. 

There is a real gap, however, between our understanding of literacy and our ability 
to assess or profile the literacy development of children. Most of the assessment 
instruments currently used by schools to collect accountability data are poor measures. 
They have little construct validity in the face of the complex nature of literacy. For 
example, NAEP’s (National Assessment of Educational Progress) reading and writing 
tests at the 4th and 8th grades collect a very limited kind of information, and yet NAEP 
publishes statistics showing that about one-third of the nation’s children are “below basic” 
readers (Bruce, Osborn & Commeyras, 1991). How should this be interpreted? Are 33% 
of all children deficit readers? 

It is common practice in the assessment field to look for deficiencies. 
Unfortunately, this practice has socialized teachers, parents, and even children, to view 
some learners as lacking the intellect needed to succeed as readers and writers. Newer 
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thinking about the development of literacy challenges this idea and asserts that almost all 
learners can learn to read and write, just as almost all learners have already learned to 
speak without any particular formal instruction. And it does matter how we frame our 
assessments of literate behaviors. On the one hand, research shows that children who 
perceive themselves as having low ability produce helpless behavior (Elliot & Dweck, 
1988), are less interested in school tasks (Maclver, Stipek & Daniles, 1991), and resort to 
unproductive strategies such as half-heartedness or setting unobtainable goals (Covington 
& Berry, 1978). On the other hand, research documents that children develop more 
complex frameworks for thinking about their own literacy in classrooms operating from 
the perspective that all learners are competent (Rasinski & DeFord, 1988; Reif, 1982). 
Children asked “what kind of reader are you?” bring different criteria to bear depending 
on their classroom situation. Some bring a simple normative framework and respond “I’m 
a good reader” or “not so good,” whereas others answer from a non-judgmental, 
descriptive framework by saying something like, “I read mostly fiction or books about 
horses” (Johnston, 1990). 

Language educators are also beginning to critique “moment-in-time” sampling 
assessment procedures. One issue is that such writing samples are often decontextualized. 
Prompts written for a large group invariably miss some individual’s experiential and 
interest needs. Another issue is that one-shot assessment tells nothing about the 
individual’s developmental path. The writing for such assessments is scored on its 
conformity to a set of established standards or its status relative to age or grade level 
norms (Domico, 1993). 
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The relationship between reading and writing can inform an assessment of writing. 
There is evidence that good readers and writers often differ from poor ones in their 
understanding of text structure and text worlds. Better writers tend to be better readers; 
better writers read more than poorer writers; and better readers produce syntactically 
more complex writing than poorer readers (Moore, 1995; Belanger, 1987; Stotsky, 1984). 
Theorist working in the tradition of information processing (Flood & Lapp, 1987; Hayes 
& Flowers, 1980; Wittrock, 1983) similarly claim reading and writing draw on common 
cognitive processes to build a text world (Langer, 1986: Purcell-Gates, 1991). As Kucer 
(1985) explains it, “Reading and writing become one instance of text world production, 
drawing from a common pool of cognitive and linguistic operations” (p. 319). 

The research supports an integrated reading-writing assessment, but there are no 
models for doing this. As Moore (1995) explains, “There is no information on individual 
growth curves or rates of learning to inform our planning because we have not yet 
designed text structure studies that would yield such findings” (p. 602). In other words, 
we need to develop new schemas for looking at children’s writing in relation to what we 
know about them as readers. By comparing a student’s writing to a standard, we can 
indicate whether the student has met the goals established by the learning community. 

This information does not, however, provide any insight into why the student performs as 
he or she does. 

Researchers in the literacy field have experimented with a variety of alternative 
assessment procedures such as think alouds (Paris & Winograd, 1990) and interviews 
asking the students to talk about their perceptions of the reading and writing processes 
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(Kemp, 1987; Langer, 1986; Rhodes & Shanklin, 1993). For example the Burke Reading 
Interview asks students to talk about what they do when they come to something they do 
not know while they are reading. When students answer this, they give insight into their 
strategies for solving the most common problem encountered in reading (Goodman, 
Watson, & Burke, 1987 ). 

Miscue Analysis is another assessment procedure that gives more complex 
information. This is a procedure in which the reader reads a passage orally and the 
evaluator notes all of the errors on a copy of the text. These errors, or “miscues,” can 
later be analyzed to determine how the reader orchestrates the reading process. The 
analysis determines what strategies and cueing systems the reader uses, thereby 
illuminating the parts of written language that make sense to the reader. When the teacher 
knows what the student can do, it is much easier to provide instruction that makes sense 
to the child and stretches the reader toward new understandings (Goodman, Watson, & 
Burke, 1987). 

Miscue Analysis can yield a score as well as qualitative data. Retellings can do this 
as well. Again, there are a number of studies to turn to (Barrs, Ellis, Hester & Thomas, 
1989; Brown & Cambourne, 1986; Rhodes & Shanklin, 1993). The retelling amounts to 
simply asking the readers to recount the story or information in their own words. It is a 
direct measure of comprehension, uncomplicated by fabricated questions or writing 
requirements. The retelling can be transcribed to create a record of speech patterns as 
well. 
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Creating the Literacy Profile 

As literacy constructs evolve, so must our assessment constructs. This is not 
happening fast enough in the Professional Development Schools where I work as a teacher 
educator. As a representative of the university, I am expected to foster conditions which 
promote “best practice” in these schools. Accountability assessments are one of the 
biggest stumbling blocks I face in this situation. Teachers see these assessments as their 
guidelines for teaching. They argue that they have no choice but to teach so that children 
will do well on these tests. This limits what our prospective teachers can see in the 
classrooms where they are learning to teach. As soon as it is possible, we need to 
introduce assessments more in keeping with our current understandings of literacy as an 
active meaning making process. 

This has prompted me to begin developing a literacy assessment procedure called 
the Literacy Profile. My goal is to develop a literacy assessment tool to use with 
elementary school children that will track their literacy development better than the current 
district-mandated and state-mandated standardized tests. 

The development is being guided by three important considerations. The profile 
assessment needs: 1) to be based on competence, 2) to reflect what the literacy community 
knows about literacy; and 3) to foster new and different conversations about how to 
support learners by providing a new way to look at children’s literate behaviors. The 
ways we look determine what we can see, and we need to find ways to see more clearly 
with the children, to record their perceptions and problem-solving strategies. We can only 
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move schools toward a more thoughtful stance about literacy by demonstrating new 
possibilities. 

This paper describes the progress of the first year of work on the Literacy Profiles. 
My research assistant, Denice Haines, and I worked on selecting passages, developing a 
set of interview questions, developing writing prompts, creating scoring rubrics, writing 
assessor directions, and checking for bias and language problems in the selected texts. 

The most daunting of these tasks has been passage selection. The profile system is being 
designed for use with children from grades one to six. We are using children’s picture 
books as reading passages, and these books are assembled into text sets (6 different titles) 
at six levels: highly predictable, predictable, highly supportive, supportive, literary, and 
independent. The books were not leveled by a formula, but rather by an analysis of their 
predictability, their graphophonic regularity, their conceptual difficulty, and their 
dependence on specific background information. Good miscue analysis depends on 
providing an unfamiliar text, so the books had to be little known gems. 

After using the texts with two different groups of children in two different schools, 
we know we have more work to do on passage selection. The lowest and highest levels 
have been most challenging to find. Some of the texts need to be moved up or down a 
level. Others just need to be eliminated. For example one of the picture books has such 
dominant pictures, we suspect the children did their retellings of the story based on the 
pictures and not the print. 

Once we chose texts, we enlisted the help of undergraduate research assistants to 
prepare a typescript of each story and a retelling guide for each book to help the assessors 
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ask good questions following the reading. Denise and I also prepared a set of interview 
questions about the reading preferences and beliefs of the children. 

The first attempts at using the Literacy Profile procedure with children were 
conducted by undergraduate students in my reading methods classes. These students were 
acting as tutors to third and fourth graders for a semester in a local elementary school. 
They conducted the literacy profile assessments with their tutoring partners and other 
children, assessing a total of fifty-five students. 

When the undergraduates administered the profiles, they let the children peruse a 
collection of the books and choose one they wanted to read. We learned quickly that 
young readers choose books on the basis of the cover art, not the book’s readability. The 
teacher interns had to listen to the children read a paragraph or two from the chosen book 
and determine if they were in the right level by counting the miscues (errors). If there was 
less than one miscue for every ten words, they moved the child up to the next level of 
book. This insured that the child would do enough problem-solving to allow for miscue 
analysis. 

In this first round of data collection, the Literacy Profile procedure was conducted 
as follows. Each child worked one-on-one with a college student. The child was asked to 
orally answer the reading interview questions while the college student wrote his or her 
answers. Then the child was helped to choose a book to read. The child’s oral reading 
was audio-taped. When the child finished reading, the college student asked the child to 
retell the story and prompted the child with questions from the retelling guide to get a 
complete retelling. Then the child was asked to write and draw about the book. 
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The college students analyzed the data collected from each child. Listening to the 
tape of the child’s reading, each assessor recorded all the reader’s miscues on a typescript 
of the book. Then the miscues were analyzed to provide information about the child’s use 
of the syntactic, graphophonic, and semantic systems of language. These scores combined 
with the retelling scores were then used to report the extent to which each reader was 
effective (monitored meaning while reading and comprehended the whole of the story) and 
efficient (used the cueing systems in balance rather than over-relying on one system and 
used only the necessary amount of visual information). 

In addition to asking each child to read and retell the story of the self-selected 
picture book, the assessors asked the children interview questions about reading and asked 
the children to write and draw in response to the story. The reading interviews provided 
insight into what each child understood about reading, and the retelling, drawing, and 
writing provided a window onto the meaning the readers assigned to the text. We did not 
however assign any scores to the drawing or writing in this pilot run. 

Insights about the First Generation of the Literacy Profile 

As a first check on the validity of the score generated by the Literacy Profile, we 
asked the teachers of the children to provide a variety of information about each child 
assessed. To check how well the leveled system of texts was working, we asked the 
teachers to tell us if they believed the child read on grade level or above or below. We 
also asked the teachers to compare the writing generated during the assessment to the 
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typical writing students did in the classroom. Finally, we asked the teachers to provide the 
language arts test scores for each student. 

Using this comparative data, we were able to establish that the sequence of leveled 
books was providing a reasonable framework for the system. Children who were 
identified as reading below grade level chose books from levels lower than their more 
successful counterparts. The teachers’ data also demonstrated that the writing we 
collected when we did the Literacy Profiles was very similar to what the children did in 
their classrooms. These were not pieces that reflected the writing process, wherein a 
student plans, drafts, revises, and edits with the support of a writing community, but 
merely response writings. Most of the children just retold the story once more. 

One of the hypotheses we wanted to test was the possibility that the miscues 
analysis scores would correlate positively with the children’s reading scores on the most 
recent standardized tests . We found that there was no correlation between test scores 
and miscue scores. While both have a relationship to reading, the framework for the two 
assessments are totally different. The miscue analysis results in numerical scores, but 
these measure entirely different constructs. 

Several improvements need to be made before we do another round of assessment 
with the Literacy Profile system. We gained fascinating insight into children’s beliefs 
about reading by conducting the reading interviews, so we would like to add questions 
about writing to the interview. We also need a scheme for assessing the writing that is not 
a rubric. Rubrics are useful in scoring crafted writing, but the writing we are collecting is 
more utiliatarian. We need to decide how best to look at it. 
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Finally, we are fascinated with the drawings the children produced. They are quite 
diverse and reflect very different points of attention. Some captured the sum of the story; 
others just a scene. We want to rewrite the prompt so that we encourage metaphorical 
thinking. 

What makes the Literacy Profile better than a standardized test of reading? 

The Literacy Profile under development is better in several ways. First, it is 
designed to be more authentic than a test. To complete a Literacy Profile, we ask the 
children to read real books rather than contrived passages written just for the purposes of 
testing. Secondly, the information provided by the Literacy Profile is qualitatively 
different than the information provided by tests. The tests divide reading and language use 
into subskills such as “recognizes cause and effect” or “draws inferences” and assesses 
each of theses subskills. The profile assessment looks at how the reader applies the skills 
of reading and assesses the reader’s success in using the subsystems of language as she 
reads. In addition, the profile includes an interview which provides information about the 
reader’s beliefs about the reading process and the reader’s sense of efficacy. Given newer 
understandings of literacy development, the information provided by the profile is more 
useful. 

Another real difference between standardized tests and the profile is that the 
children are helped to choose a book that both interests and challenges them. 

Standardized tests make no allowances for developmental differences or personal 
preferences or interests. This is a serious weakness because it is impossible to gain valid 
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information about a child’s reading strategies if the child cannot read or is not motivated 
to read the passages offered. To know what kind of problem-solving skills the child 
brings to the reading process, we must provide the child with a text that falls into his or 
her reading range and interest. 

Initially, the university can train the preservice teachers in the Professional 
Development Schools to provide this assessment service for the school. Preparing 
preservice teachers to administer the Literacy Profile adds to their knowledge about the 
reading process. Conducting the assessment provides first hand experience in analyzing 
reading from a more informed perspective. In addition, teachers in the schools can also be 
educated by this reading and writing assessment, which can, in turn, change how they 
support readers in their classrooms. 

As Meyers (1996) points out, we are “changing our minds” about literacy— what it 
is an how we use it. We desperately need assessment tools that reflect some of the 
changes we understand in theory, but struggle to introduce into the classroom of public 
schools. 
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